A. Field of the Invention
The present invention relates to solid state electronic image sensors and in particular to methods for detecting a moving object in a scene using exposures of an image sensor. The methods are particularly suitable for application to airborne imaging systems, such as military aerial reconnaissance image sensors.
B. Description of Related Art
Image sensors for capturing electronic representations of images in the visible wavelengths have been around since about 1970, when the silicon charge coupled device (CCD) was invented. Over the years, CCDs have become the standard for visible spectrum electronic imaging, replacing film in most applications. Various architectures have been developed to make CCD operation more effective for the specific application. Such architectures include the full frame, frame transfer, and interline transfer CCD.
The processes related to silicon CCD manufacturing have always been reasonably expensive and complex, with the implementation technology of choice being N type metal oxide (NMOS). The associated costs and operational characteristics of NMOS CCDs, and later P type (PMOS)CCDs, eventually persuaded designers to seek an image sensor solution using the more easily manufactured Complimentary Metal Oxide Semiconductor (CMOS) technology. CMOS had become the technology of choice utilized by most foundries for making computer integrated circuits.
The advantages of using CMOS for image sensors included having a wider selection of foundries, lower manufacturing costs, lower power/voltage operation, and especially the ability to add peripheral CMOS circuitry on the image sensor substrate, the later feature being attractive for developing an image sensor on a single chip. CMOS imagers therefore could have active circuitry added within the picture sensing element structure (pixel). These imagers became known as Active Pixel Sensors (APS). The APS CMOS image sensor (CIS) became feasible approximately ten years ago when CMOS processing technology advanced to the point of enabling minimum feature sizes small enough to meet the required pixel circuit density, while maintaining adequate optical aperture for a small pixel (diffraction limited) visible image sensor.
As CMOS image sensor technology continues to progress, the comparison of performance between the CCD and the CMOS image sensor has become an important topic of discussion by designers and end users. Today, CMOS imager development has yielded a sensor with characteristics close to that of the CCD, but still lacking in some respects. Overall, the CMOS APS has the advantage of lower cost and design flexibility, but has yet to fully match the performance of the CCD.
Currently, foundry-based CMOS image sensor technologies largely focus on manufacturing of low cost visible sensors for use in high volume consumer products, such as cell phones, PC cameras, toys, automotive sensors, camcorders and low cost SLR cameras. Consequently, the CMOS manufacturing technologies are largely the same as those used in manufacturing of high volume computer memory or mixed signal products. Few, if any, high volume commercial grade CMOS imaging products are manufactured using uniquely customized image sensor specific processes or complex pixel design technologies.
Although some CMOS imager foundries have begun to support unique process modifications to create specific process and design features for application specific integrated circuit (ASIC) clients, in nearly every case the custom CMOS image sensor technologies that enable these high performance applications are unique to the ASIC client, and not accessible to other users.
As will be explained in further detail below, improvements are disclosed in an APS CMOS imager which improve the dynamic range of the CMOS image sensor such that it meets or exceeds current customer expectations for imager sensitivity, response and signal to noise ratio such that it can be used for airborne imaging, especially in a military reconnaissance application. Such an imager, when deployed in an aerial reconnaissance camera, can be operated in accordance with the disclosed methods to detect moving objects (e.g., vehicles) in a scene.
The performance requirements for aircraft based Visible and Near Infra-Red (Vis and NIR) wavelength imaging systems mandate that solid state Vis/NIR focal plane technology enable unique capabilities unlike most commercial or consumer applications of imaging technology. Historically, CCD image sensor technology has provided the needed critical performance and configuration demanded by airborne imaging applications. However, as airborne reconnaissance systems begin to incorporate additional functionality and interoperability through reductions in size, weight, power, and cost, it is becoming increasingly difficult for CCD technology to support these objectives, due to limitations in the fundamental fabrication and operational principles of the CCD image sensor itself. Although present CMOS image sensor technology does not support the broad, demanding performance requirements currently provided by the CCD, the successful evolution of airborne reconnaissance imaging systems will rely on the adaptation of CMOS image sensor technology to enable the improvements necessary to advance the art. The CMOS imager design aspects disclosed herein enable this result to be achieved.
In order to achieve both the stringent and contradictory performance requirements of the airborne reconnaissance grade Vis/NIR image sensor in a CMOS technology, customization of both the CMOS process and the pixel design technologies is required. To further understand the problem, it will be noted that optimum sensor performance requirements include seemingly contradictory aspects. On the one hand, each pixel in the focal plane must have a large full well electron capacity for shot noise (thermal) limited high background, large signal performance, as would be expected during the daytime with full scene illumination. However, on the other hand, each pixel must have high sensitivity and low noise for small signal extraction, in order to provide for reconnaissance capability in low light level situations, such as when shadows are present in the field of view. Therefore, aspects of the present disclosure provide for a novel, modified pixel design to satisfy this dynamic range dilemma.
Furthermore, present CMOS imaging technologies are optimized for photoresponse in the visible band from 450 nm to 700 nm, with the desired NIR response filtered or dramatically reduced. Customization of the CMOS process is therefore also needed to further extend the sensor response from 700 nm to 950 nm for airborne reconnaissance applications.
The embodiments disclosed solve the contradictory dynamic range dilemma of the airborne CMOS image sensor by providing a pixel design, which achieves large well fill performance as well as high sensitivity and a high signal to noise ratio (SNR) while preserving linearity of response.
It is known in the art that CMOS active pixel sensors rely on each pixel having a charge to voltage conversion amplifier to create local voltages representative of the signal value illumination recorded at the pixel. A representative prior art CMOS pixel 10 is shown in FIG. 1A. Incident radiation 12 impinges on a photodetector 14. Collected charge is supplied to an amplifier 16. A gate 18 is clocked to direct a voltage signal on the output of the amplifier to an output multiplexer (not shown) along conductor 24. A reset gate 20 resets a transistor 22 at the output of the photodetector 14 as shown.
The typical prior art (FIG. 1A) teaches that a CMOS APS pixel contains one detector 14 and one amplifier circuit 16 per pixel. The performance of the pixel is determined by the detection, amplification, and noise performance of the single detector, single amplifier combination. Typically, most imaging applications will either fall in the category of still or video photography, or controlled light source photography. Prior art pixel design has not previously been required to provide both high sensitivity and signal to noise ratio for small signals while simultaneously providing good contrast signal to noise for small signals buried within high background illumination scenes.
FIG. 2 is an illustration of a CMOS image sensor 30 having an array 32 of CMOS pixel sensors 10, one of which is shown enlarged. The array 32 is arranged in rows and columns of pixels, perhaps 25 or 50 million in all. FIG. 2 shows a column amplifier 36 which amplifies the output voltage from the pixel sensors 10. The amplified signals are supplied to a multiplexer 34. The multiplexed output signals are amplified by an amplifier 38. The amplified signal is supplied to A/D converters and signal processing circuitry which is conventional.
FIG. 3 is an illustration of a four transistor prior art CMOS Active Pixel sensor. This design includes a global shutter transistor 40, reset transistor 22, amplifier transistor 16 and select transistor 18. By controlling the biasing of the shutter transistor 40, the accumulated charge on the photodiode detector 14 is selectively applied to the charge to voltage conversion amplifier transistor 16.
FIG. 4 is another prior art four transistor CMOS pixel design. It includes a transfer transistor 42, a reset transistor 44, charge to voltage conversion amplifier 16 and a select transistor 18 transferring the voltage signal from the output of the amplifier 16 to the conductor 24. The transfer gate transistor 42 acts in a similar fashion to the shutter transistor 40 of FIG. 3.
Prior art pixel design has addressed the extension of CMOS pixel dynamic range to accommodate large background signals by compressing the response to large signals with non-linear amplification or by subtracting signal values through down stream signal processing. The compressed response to large signals degrades the contrast signal to noise ratio for the airborne reconnaissance imaging application by reducing the contrast between the small signals present in the large background signal and the background signal itself. The associated graph shown in FIG. 1B illustrates the compromised signal handling capacity and the non linearity of the response as the illumination increases and is compressed. For example, in region 26, there is very little increase in the output signal on conductor 24 as illumination increases.
Various other methods have been employed to compress the input dynamic range including the utilization of variable detector voltage values, multiple storage of lines of signal integrated with different integration times then subtracted, logarithmic response amplifiers, anti-blooming structures to limit collected signal charge, pixel level analog to digital conversion (ADC) and gain adjustment, as well as other concepts.
Typically, the response of a visible detector is linear with respect to optical signal input to the point of either pixel well saturation or full signal swing of the detector sense node. Beyond the saturation illumination level, the response curve becomes essentially flat. See FIG. 1B. The prior art methods to extend detector dynamic range do not preserve the linear response of the detector while maintaining signal integrity for both high and low scene illumination levels.
The full well capacity of a CCD image sensor pixel is generally limited by the charge handling capacity of a gated potential well that usually has an antiblooming feature to avoid signal charge from mixing with adjacent pixels when a pixel that has reached its well saturation limit. Conversely, the saturation limit of a CMOS image sensor is typically more a complex function of the floating diffusion sense node voltage swing, photodiode storage capacity, and additional capacitances added to the pixel amplifier circuit. The critical relationship for the saturation value of a CMOS sensor pixel is CV=Q, where V=the useful linear voltage swing allowed at the detector sense node (reset node typically), C=the capacitance of the sense node (amplifier 16 input node) (including all parasitics), and Q=all photogenerated and thermally generated signal electrons collected and incident on the sense node.
The voltage limit at the sense node is determined by the supply voltage and the threshold of the reset transistor 22 of FIG. 1A. The largest voltage swing at the sense node for an N-channel transistor is Vdd−Vt. The practical reality is that the sense node is typically coupled to the gate of a source follower circuit, which has a range of useful input voltages for operation. Typically, the maximum useful voltage swing at the sense node of a CMOS image sensor is <60% of Vdd.
In general, the P/N junction diode photodetectors have a much higher limit for charge storage capacity than the voltage swing at the sense node will allow. The photodiode typically begins to forward bias when over-illuminated such that excess charge is swept into the substrate, thus creating a built-in antiblooming circuit.
The relationship that describes the conversion of signal charge to useful voltage signal is determined by the sense node capacitance or qe/Cnode (microvolts per electron). As the node capacitance increases, the charge to voltage gain decreases, but the dynamic range of the allowable input signal increases as Vmax is reached more slowly.
Increasing the node capacitance is the simple method for increasing the saturation charge limit for the CMOS sensor pixel. Increasing the maximum voltage swing in the sense node is another method. Unfortunately, the maximum voltage swing is limited by the maximum allowable Vdd voltage for the CMOS process design rules being used. Smaller dimension design rules dictate smaller voltages. For circuits that require small geometries for packing density and/or low voltages for low power operation, the maximum Vdd supply value scales to ever-lower values.
The effect of increasing the capacitance at the sense node is to decrease the signal to noise ratio by 1/√C. Hence, for small signals, it is desirable to minimize the sense node capacitance. However, in the airborne imaging and reconnaissance application, it is also necessary to have a large charge handling capacity to improve shot noise limited signal to noise ratio (varies as Vsignal/√Vsignal) for scenes with high background signal and low contrast between target features and the background signal. For the case of optimized small signal gain, adding gain (g) to the sense amplifier while decreasing the sense node capacitance results in an increase in the signal to noise ratio proportional to √g/√C (in simplistic terms).
A critical requirement for airborne reconnaissance imaging systems is to have high signal to noise ratios, for both small signal and high background signal imaging conditions. As previously stated, this creates a conflict for conventional pixel design. Small node capacitance favors better small signal SNR, while large node capacitance favors better SNR for large signals. The sensor designs described herein achieve both SNR goals by utilizing novel CMOS image sensor pixel designs.
Further background art includes the paper “Large Area TDI Image Sensor for Low Light Level Imaging,” M. Farrier and R. Dyck, IEEE Transactions on Electron Devices, Vol. ED-27, No. 8 (August 1980), U.S. Pat. Nos. 6,486,504; 6,011,251; 6,757,018; 6,734,905; and 6,710,804. See also Wide Intrascene Dynamic Range CMOS APS Using Dual Sampling, O. Y. Pecht, et al., IEEE Transactions on Electron Devices, Vol. 44, No. 10 (October 1997).
In aerial reconnaissance applications, it is desirable to detect moving objects in a scene. Such objects could be for example a military vehicle in a battlefield scene. This disclosure provides for methods for identifying a moving object in a scene. The methods are particularly useful in an advanced CMOS imaging sensor which provides for both low small and high background imaging conditions.